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DETAILED ACTION 



Claim Rejections -35 USC § 103 
The text of those sections of Title 35, U.S. Code not included in this action can be found 
in a prior Office action. 

1. Claims 1, 12, 14, and 23 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Schein et al (US Patent No. 6,075,575), hereinafter referred to as Schein, in view of Buil et al 
(US Patent No. 6,718,307), hereinafter referred to as Buil and further in view of White et al (US 
Patent No. 6,408,272). 

2. As per claims 1, 14, 23, Schein et al. disclose a system/method comprising: a speech 
recognizer that recognizes spoken requests for television programming information, (see col. 6, 
lines 12-16). Schein fail to explicitly teach a state machine having three states including a first 
state in which the state machine is not listening, a second state in which the state machine listens 
for only one query, and a third state in which it is listening to all queries, and using the state 
machine to provide inputs to a speech recognizer. However, implementation of listening states 
in a speech recognizer was well known in the art. ; 

In a similar field of endeavor, Buil discloses a speech input based input device which has 
a standby mode in which it can only recognize an activation instruction and an operation al mode 
in which it can recognize commands for control of an apparatus, which reads on the three state 
listening machine since if the recognizer is off or the system is processing the user's input 
provides a state in which the machine is not listening, the standby mode to recognize an 
activation instruction provides a state in which the machine is listening for only one query, and 
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the operational mode to recognize commands for controlling an apparatus provides a state in 
which the machine is listening to all queries (col. 4, lines 19-59). Buil specifically teaches the 
multiple mode/state of the system reduces the chance of sending unintentional commands to the 
apparatus (col. 5, lines 3-14). 

Therefore, it would have been obvious to one of ordinary skill at the time of the 
invention to modify the system of Schein to implement a multi-mode recognition system, as 
taught by Buil, for the purpose of reducing the chance of sending unintentional commands to the 
apparatus, as suggested by Buil. 

Schien does not explicitly teach if the user makes a new utterance while the state 
machine is in the third state and is working on a previous utterance by the user, the state machine 
stops working on the user's previous utterance and begins working on the user's new utterance. 
However, implementation of barge-in capabilities was well known in the art. White discloses a 
voice user interface with a barge-in component (col. 15, lines 40-52), which upon detection of 
the user's speech, halts the current VUI processing to attend to the new speech input. It would 
have been obvious to one of ordinary skill at the time of the invention to modify the system of 
Schein to implement a barge-in component as suggested by White, for the purpose of providing 
prompt and efficient recognition and command execution to the user of the system. 
3. As per claim 12, Schein, Buil, and White disclose everything as claimed in claim 1 . 
Schein further disclose a system including a television coupled to a set top box and a remote 
control (see col. 3, lines 21-24). 
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4. Claim 1 1 is rejected under 35 U.S.C. 103(a) as being unpatentable over Schein in view of 
Buil and White, as applied to claim 1 above, and further in view of well known prior art. 

As per claim 1 1 Schein, Buil, and White disclose everything as claimed in claim 1 . 
Schein fails to explicitly teach a system including a processor coupled to the speaker and a 
microphone, the output of said speaker being subtracted from the output of said microphone to 
reduce interference. However, subtracting undesired signals from a speech signal input to a 
microphone was well known in the art. 

Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to subtract the output of the speaker from the input of the microphone, as 
was well known in the art, for the purpose of improving the operation of the system by reducing 
interference, and thereby improving the performance and accuracy of the speech recognizer. 

As per claim 31, Schein, Buil, and White disclose everything as claimed in claim 1. 
Schein fails to explicitly teach a system including 

5. Claims 13, 22, and 30-33are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Schein in view of Buil and White, as applied to claims 1,14, and 23 above, in view of Tomitsuka 
et al (US Patent No. 5,566,271), hereinafter referred to as Tomitsuka, and further in view of 
Mueller et al (US Patent No. 6,009,398), hereinafter referred to as Mueller. 

6. As per claims 31-33, Schein, Buil, and White disclose everything as claimed in claims 1, 

14, and 23. Additionally, Schein discloses an output device that generates response to spoken requests from 
television programming information, (see Fig. 4A). 
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Schein fail to explicitly teach a system including a module coupled to said recognizer to 
implement conversational speech. However, this feature was well known in the art. 

In a similar field of endeavor, Tomitsuka disclose a system comprising a voice synthesis 
module for implementing conversational speech, (see Fig. 1, block 19). 

Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to use a voice synthesis module in Schein, as taught by Tomitsuka, for the 
purpose of using a voice synthesis in conjunction with a voice recognition module to improve 
man-machine interaction, as suggested by Tomitsuka. 

Additionally, Schein discloses a graphical user interface which provides information in a 
visual form about television programming and a voice user interface which responds to voice 
requests from the user..., (see Fig. 4A and col. 6, lines 36-42). 

Schein does not specifically disclose the graphical user interface and voice user interface 
communicating such that the focus of one is communicated to the other. However this feature 
was well known. 

In a similar field of endeavor, Mueller discloses a system, which allows for user and 
system interaction via speech, visual and auditory interfaces. Specifically, at col. 4, line 56 
continuing to col. 5, line 29, Mueller teaches implementation of a context analyzer which 
coordinates the speech interactions with the visual interface to accommodate how the user 
interacts with system when the user provides input via speech or via a text based graphical user 
interface. 

Therefore, it would have been obvious to ohe of ordinary skill at the time of the invention 
to implement interface focus communication as taught by Mueller, for the purpose of ensuring 
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system performance when the user provides input via a visual or graphical user interface, as 

suggested by Mueller. 

As per claims 13, 22, and 30, Schein, Buil, White, Tomitsuka, and Mueller disclose 
everything as claimed claims 1,14 and 23. Schein fail to explicitly teach a system wherein the 
output device is a speech synthesizer that generates responses. However, this feature was well 
known in the art. 

Tomitsuka discloses a system wherein the output device is a speech synthesizer that 
generates responses (see Fig. 1, block 19). 

Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to use a voice synthesis module as taught by Tomitsuka in the system of 
Schein, for the purpose of improving man-machine interaction, as suggested by Tomitsuka. 

7. Claims 4-8, 17-19, and 26-28 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Schein in view of Buil, White, Tomitsuka and Mueller, as applied to claims 31-33 above, 
and further in view of Haddock (US Patent No. 5,265,014). 

8. As per claims 5, 18, and 27, the combination of Schein, Buil, White, Tomitsuka, and 
Mueller teach everything as claimed in claims 31-33. However, the combination fails to disclose 
producing a select variable and a where variable from a query received from a user. However, 
this feature was well known in the art. 

In a similar field of endeavor, Haddock teaches the process of how an ambiguous query 
is processed and syntactically analyzed to develop a representation of the syntactic structure. 
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The system produces a syntactic structure based upon a question/sentence format to represent a 
set of questions, a set of nouns, a set of phrases, and a set of verbs. Additionally, at col. 6, lines 
29-41 Haddock discloses that the system characterizes the query to find values of the variable 
that fulfills the condition or question within the query. In the example provided by Haddock, the 
query produced the syntactic structure (WHICH X s.t. (PAINT *REF(he) X)), and the request of 
the query is to find any values of X that meet the condition of (PAINT *REF(he) X). The 
condition (PAINT REF(he) X), is comprised of a semantic predicate PAINT and two variables, 
the painter (REF(he)) and the painting (X), which reads on "produces a select variable and a 
where variable from a query received from a user." 

Therefore, it would have been obvious to one of ordinary skill at the time of the 
invention to modify the system of Schein to process the query into a syntactic structure with 
different variables, as taught by Haddock, for the purpose of characterizing the query to find 
values of the variable that fulfills the condition or question within the query. 

As per claims 4, 6, 17, 19, 26, and 28, the combination of Schein, Buil, White, 
Tomitsuka, and Mueller teach everything as claimed in claims 31-33. However, the 
combination fails to teach the implementation of historical information to modify the meaning 
from the users input query. However, this feature was well known in the art. 

At col. 6, lines 48-60, Haddock discloses that in the example query, the meaning of the 
pronoun "he" is ambiguous and in order to resolve the ambiguity for further processing, the 
system uses the history of the dialog of a previous query to determine the reference to the "he", 
in order to form the completed query of (WHICH X s.t. (PAINT DEGAS X)), wherein the value 
of the *REF(he) is obtained from a previous query. 
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Therefore, it would have been obvious to one of ordinary skill at the time of the 
invention to modify the system of Schein to implement historical information to modify the 
meaning of the input of the user's input query, as taught by Haddock, for the purpose of 
resolving the ambiguity in the input query. 

As per claim 7, Schein, Buil, Tomitsuka, Mueller and Haddock teach everything as 
claimed in claim 6. However, Schein fails to teach determining whether the query includes both 
first and second type of variable and if so, not using the historical information to modify the 
user's input query. 

At col. 6, lines 39-59, Haddock discloses the functionality of the system for determining 
the meaning of an ambiguous query, in which the query representation contains the pronoun 
"he". The query is ambiguous because "it is not yet known who the pronoun "he" refers to 
because that information lies outside the query" (col. 6, lines 45-47). In this instance the system 
uses the history information to determine to whom "he" refers. However, if the utterance 
contains both of the attributes of the utterance, there is no ambiguous query and there is no need 
to use the history vector, as indicated in col. 5, lines 33-42, in which the query provided includes 
the names of the specific painter of whom a user wishes to retrieve information (query 1 and 
query 2), which reads on "determining whether the query includes both first and second type of 
variable and if so, not using the historical information to alter the meaning derived from the 
speech recognizer." 

Therefore, it would have been obvious to one of ordinary skill at the time of the invention 
to modify the system of Schein, to implement determining whether the query includes both a first 
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and a second type of variable, for the purpose of reducing unnecessary processing if the meaning 
of the query can be ascertained. 

As per claim 8, Schein, Buil, White, Tomitsuka, Mueller and Haddock teach everything 
as claimed in claim 6. However, Schein fails to te^ch determining whether the query includes 
only one of two variable types, and if so, using the -historical information to modify the user's 
input query. 

At col. 6, lines 39-59, Haddock discloses the functionality of the system for determining 
the meaning of an ambiguous query, in which the query representation contains the pronoun 
"he". The query is ambiguous because "it is not yet known who the pronoun "he" refers to 
because that information lies outside the query" (col. 6, lines 45-47). In this instance the system 
uses the history information to determine to whom "he" refers. However, if the utterance 
contains both of the attributes of the utterance, there is no ambiguous query and there is no need 
to use the history vector, as indicated in col. 5, lines 33-42, in which the query provided includes 
the names of the specific painter of whom a user wishes to retrieve information (query 1 and 
query 2). At col. 6, line 13, Haddock specifically states that query 2 is ambiguous because of the 
pronoun "he", and thus the system must resolve the ambiguity of the query via the history 
information. At col. 7, lines 10-35, Haddock refers to the ambiguous fourth query from col. 5, 
lines 42-44, in which "these" is used in the query. The system uses history information to 
determine the reference of the set or subset of paintings "these" actually refers. Haddock's 
determination of needing to use the history dialog to resolve an ambiguous query in one instance 
or not needing to use a history dialog in another instance would suggest and/or motivate one of 
ordinary skill in the art to specifically determine if an utterance includes all necessary variables. 
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Additionally, Haddock discloses in order to form the completed query of (WHICH X s.t. 
(PAINT DEGAS X)), the value of the *REF (he) is obtained from a previous query, which 
reads on "merge the variable with the historical information to derive a meaning from the 
request." 

Therefore, it would have been obvious to one of ordinary skill at the time of the 
invention to modify the system of Schein to implement determining if an input query includes 
only one of a type of variable, and if so using the history information to form a completed 
query, as taught by Haddock, for the purpose of reducing ambiguity in the query, as suggested 
by Haddock. 

9. Claims 9-10, 20, and 29 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Schein in view of Buil, White, Tomitsuka and Mueller, as applied to claims 31-33 above, and 
further in view of Houser et al (US Patent No. 5,774,859). 

10. As per claims 9-10, 20, and 29, the combination of Schein, Buil, White, Tomitsuka, and 
Mueller teaches everything as claimed in claims 31-33. However, the combination does not 
form time attributes in a request. However, this feature was well known. 

In a similar field of endeavor, Houser teaches an information system with a speech 
interface which controls a device such as a television and access to broadcast information. 
Specifically, at col. 30, lines 54-64 and col. 31, lines 39-43, Houser implements processing of 
search requests for television broadcast information based on time ranges ("only within 4 hours" 
or "only after eleven AM"). At col. 30, lines 56-58, Houser teaches that implementation of time 
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information commands will limit the search to programming information within an identified 
range. 

Therefore, it would have been obvious to one of ordinary skill at the time of the invention 
to modify the system of Schein and implement time command information as taught by Houser, 
for the purpose of limiting the search to programming information within an identified range, 
and therefore decrease system response time and ehhance system performance. 

11. Claim 21 is rejected under 35 U.S.C. 103(a) as being unpatentable over Schein in view of 
Buil, White, Tomitsuka and Mueller, as applied to claim 32 above, and further in view of well 
known prior art. 

12. As per claim 21, Schein, Buil, Tomitsuka, and Mueller disclose everything as claimed in 
claim 32. Schein fails to explicitly teach a system including a processor coupled to the speaker 
and a microphone, the output of said speaker being subtracted from the output of said 
microphone to reduce interference. However, subtracting undesired signals from a speech signal 
input to a microphone was well known in the art. 

Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to subtract the output of the speaker from the input of the microphone, as 
was well known in the art, for the purposeof improving the operation of the system by reducing 
interference, and thereby improving the performance and accuracy of the speech recognizer. 
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13. Claim 31 is rejected under 35 U.S.C. 103(a) as being unpatentable over Schein in view of 
Buil and White, as applied to claim 1 above, and further in view of Gould (US Patent No. 
6,101,468) 

As per claim 31, Schein, Buil, and White disclose everything as claimed in claim 1. 
Schein fails to explicitly teach a system including setting a flag when a speech variable 
originated from the user's utterance and a second flag when a speech variable has been confirmed 
audibly to the user. However, implementation of confirmation flags were well known in the art. 
In a similar field of endeavor, Gould discloses apparatuses and methods for training and 
operating speech recognition systems which recognizes a user's utterance, presents the 
recognized utterance to the user and sets a flag if the user has confirmed the presented utterance 
as the actual token/word (col. 13, lines 18-35). Gould specifically teaches setting the flag 
provides the system with an indication the particular token is properly labeled and is safe to use 
for adaptive training. It would have been obvious to one of ordinary skill at the time of the 
invention modify the system of Schein to implement the confirmation flags of Gould, so as to 
provide the best tokens of speech that can be used in adaptively training the recognizer, thereby 
improving the accuracy and performance of the speech recognizer. 

Response to Arguments 

14. Applicant's arguments with respect to claims 1,4-14,17-23, and 26-34 have been 
considered but are moot in view of the new ground(s) of rejection. 
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Conclusion 

Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Angela A. Armstrong whose telephone number is 571-272-7598. 
The examiner can normally be reached on Monday- Thursday 1 1 :30-8:00 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, David Hudspeth can be reached on 571-272-7843. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. : Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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Primary Examiner 
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